Skip to content

Add MAUI iOS Inner Loop measurements for CI#5187

Draft
davidnguyen-tech wants to merge 100 commits into
dotnet:mainfrom
davidnguyen-tech:nguyendav/maui-ios-inner-loop
Draft

Add MAUI iOS Inner Loop measurements for CI#5187
davidnguyen-tech wants to merge 100 commits into
dotnet:mainfrom
davidnguyen-tech:nguyendav/maui-ios-inner-loop

Conversation

@davidnguyen-tech
Copy link
Copy Markdown
Member

Summary

Adds MAUI iOS Inner Loop performance measurements to CI, supporting both iOS simulators and physical devices.

What's included:

  • iOSInnerLoopParser.cs — Binlog parser extracting iOS build task/target timings
  • Startup.cs/Reporter.cs — Wiring and null-safety fix
  • ioshelper.py — iOS simulator and physical device management
  • runner.py — IOSINNERLOOP execution branch (build → deploy → measure → incremental)
  • Scenario scripts — pre.py, test.py, post.py, setup_helix.py
  • maui_scenarios_ios_innerloop.proj — Helix workitem definition (simulator + physical device)
  • Pipeline YAML — Job entries in sdk-perf-jobs.yml and routing in run_performance_job.py

Measurements:

  • First deploy: full build + install + launch timing via binlog parsing
  • Incremental deploy: source edit → rebuild + reinstall + relaunch timing

Targets:

  • iOS Simulator (iossimulator-arm64) on macOS Helix machines
  • Physical iPhone (ios-arm64) on Mac.iPhone.17.Perf queue

Based on:

  • Existing Android Inner Loop CI scenario (working reference)
  • Local iOS implementation from feature/measure-maui-ios-deploy branch

davidnguyen-tech and others added 30 commits April 3, 2026 15:54
- Create iOSInnerLoopParser.cs: binlog parser for iOS inner loop build
  timings, extracting iOS-specific tasks (AOTCompile, Codesign, MTouch,
  etc.) and targets (_AOTCompile, _CodesignAppBundle, _CreateAppBundle,
  etc.) plus shared tasks (Csc, XamlC, LinkAssembliesNoShrink)

- Wire into Startup.cs: add iOSInnerLoop to MetricType enum and map it
  to iOSInnerLoopParser in the parser switch expression

- Fix Reporter.cs: guard against null/empty PERFLAB_BUILDTIMESTAMP to
  prevent ArgumentNullException on DateTime.Parse(null) when the env
  var is unset (falls back to DateTime.Now)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- const.py: Add IOSINNERLOOP constant and SCENARIO_NAMES mapping
- ioshelper.py: New module with iOSHelper class for simulator and physical
  device management (boot, install, launch, terminate, uninstall, find bundle)
- runner.py: Add iosinnerloop subparser, attribute assignment, and full
  execution branch (first build+deploy+launch, incremental loop with source
  toggling, binlog parsing, report aggregation, and Helix upload)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
pre.py: Install maui-ios workload, create MAUI template (no-restore for
Helix), strip non-iOS TFMs with flexible regex, inject MSBuild properties
(AllowMissingPrunePackageData, UseSharedCompilation), copy merged
NuGet.config for Helix-side restore, create modified source files for
incremental edit loop, check Xcode compatibility.

test.py: Thin entrypoint that builds TestTraits and invokes Runner.

post.py: Uninstall app from simulator, shut down dotnet build server,
clean directories.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Create the Helix machine setup script for MAUI iOS inner loop measurements.
This script runs on the macOS Helix machine before test.py and handles:

1. DOTNET_ROOT/PATH configuration from the correlation payload SDK
2. Xcode selection — auto-detects highest versioned Xcode_*.app, matching
   the pattern used by maui_scenarios_ios.proj PreparePayloadWorkItem
3. iOS simulator runtime validation via xcrun simctl
4. Simulator device boot with graceful already-booted handling
5. maui-ios workload install using rollback file from pre.py, with
   --ignore-failed-sources for dead NuGet feeds
6. NuGet package restore with --ignore-failed-sources /p:NuGetAudit=false
7. Spotlight indexing disabled via mdutil to prevent file-lock errors

Follows the same structure as the Android inner loop setup_helix.py:
context dict pattern, step-by-step functions, structured logging to
HELIX_WORKITEM_UPLOAD_ROOT for post-mortem debugging.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Define the Helix .proj file for iOS inner loop measurements, modeled after
the Android inner loop .proj and existing maui_scenarios_ios.proj patterns.

Key design decisions:
- Build on Helix machine (not build agent) because deploy requires a
  connected device/simulator. PreparePayloadWorkItem only creates the
  template and modified source files via pre.py.
- Workload packs stripped from correlation payload (RemoveDotnetFromCorrelation
  Staging) and reinstalled on Helix machine by setup_helix.py.
- Environment variables set via shell 'export' in PreCommands (not in Python)
  because os.environ changes don't persist across process boundaries.
- No XHarness — iOS inner loop uses xcrun simctl directly.
- Simulator-only for now; physical device support (ios-arm64, code signing)
  is structured as a future TODO pending runner.py device support.
- 01:30 timeout to accommodate iOS build + workload install + NuGet restore.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- sdk-perf-jobs.yml: Add Mono Debug job entry for maui_scenarios_ios_innerloop
  on osx-x64-ios-arm64 (Mac.iPhone.17.Perf queue)
- run-performance-job.yml: Add maui_scenarios_ios_innerloop to the in() check
  so --runtime-flavor is forwarded to run_performance_job.py
- run_performance_job.py: Add maui_scenarios_ios_innerloop to
  get_run_configurations() (CodegenType, RuntimeType, BuildConfig) and to the
  binlog copy block for PreparePayloadWorkItems artifacts

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- ioshelper.py: Add detect_connected_device() with auto-detection via
  xcrun devicectl (JSON + fallback text parsing), uninstall_app_physical,
  terminate_app_physical, close_physical_device, and cleanup() dispatch
- runner.py: Add --device-type arg (simulator/device) to iosinnerloop
  subparser, auto-infer from RuntimeIdentifier, auto-detect device UDID,
  branch setup/install/startup/cleanup for physical vs simulator
- setup_helix.py: Detect device type from IOS_RID env var, skip simulator
  boot for physical device, add detect_physical_device() for Helix
- post.py: Handle physical device uninstall via devicectl with UDID
  auto-detection fallback
- maui_scenarios_ios_innerloop.proj: Add physical device HelixWorkItem
  (conditioned on iOSRid=ios-arm64), pass IOS_RID to Pre/PostCommands,
  add --device-type arg to both simulator and device workitems

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e sort

Fix 1 (Major): Replace non-existent 'devicectl terminate --bundle-id' with
'--terminate-existing' flag on launch command. Make terminate_app_physical()
a no-op with documentation explaining why.

Fix 2 (Medium): Write devicectl JSON output to temp file instead of
/dev/stdout, which mixes human-readable table and JSON. Applied in both
ioshelper.py and setup_helix.py with proper temp file cleanup.

Fix 3 (Medium): Add standard UUID pattern (8-4-4-4-12) to UDID regex in
_detect_device_fallback() for CoreDevice UUID format compatibility.

Fix 4 (Medium): Normalize MAUI template to always use Pages/ subdirectory
in pre.py. If template puts MainPage files at root, move them to Pages/.
Add explanatory comment in .proj documenting the coupling.

Fix 5 (Minor): Use tuple-of-ints version sort for Xcode selection instead
of string comparison (fixes 16.10 < 16.2 ordering bug).

Fix 6 (Minor): Make simulator boot failure fatal with sys.exit(1). Add
dynamic fallback to latest available iPhone simulator before failing.

Fix 7 (Nit): Add missing trailing newline to runner.py.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace deprecated tempfile.mktemp() with tempfile.mkstemp() in both
  ioshelper.py and setup_helix.py to avoid TOCTOU race condition.
- Fix unreachable fallback in detect_connected_device(): when devicectl
  exits non-zero (e.g., older Xcode without --json-output), call
  _detect_device_fallback() instead of returning None immediately.
- Guard against missing JSON report in runner.py IOSINNERLOOP branch:
  Startup.cs only writes reports when PERFLAB_INLAB=1, so local runs
  would crash with FileNotFoundError. Now degrades gracefully with
  empty counters and a warning.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Temporarily disable all other scenario jobs to speed up CI
iteration while validating the new MAUI iOS Inner Loop scenario.
This change should be reverted before merging.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Capture dotnet build output instead of crashing on CalledProcessError
- Create traces/ directory before first build
- Fix setup_helix.py to write output.log (matches .proj expectation)
- Improve error handling for build failures

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The dotnet build stdout/stderr wasn't appearing in Helix console logs,
making it impossible to diagnose build failures. Explicitly capture and
print build output through Python logging.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Build 2943141 hit the 90-minute timeout. iOS first build with AOT
compilation can take 30+ minutes, plus 3 incremental iterations.
Increasing to 2.5 hours to allow full completion.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Helix machines have Xcode 26.2 but the iOS SDK requires 26.3.
The minor version difference shouldn't affect build correctness,
so bypass the check with ValidateXcodeVersion=false.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Mac.iPhone.17.Perf queue uses Intel x64 machines which need
iossimulator-x64, not iossimulator-arm64. Add architecture
detection in setup_helix.py and update default RID in .proj.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The traces upload directory already exists from the first build,
causing copytree() to fail on subsequent iterations. Clear it
before each parsetraces() call.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The -v:n flag was added to debug build errors but produces
excessive file copy logs. Default verbosity shows errors/warnings.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add /p:MtouchLink=None to disable managed linker for Debug inner
  loop builds, avoiding MT0180 errors on machines without Xcode 26.3
- Add minimum Xcode version check in setup_helix.py for fast failure
  with clear diagnostics when machine has Xcode < 26.0

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove temporary ${{ if false }}: wrappers that disabled all jobs
except iOS inner loop during iterative CI debugging.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…loop

- Separate install from simulator/device setup in ioshelper.py
- Capture install time for first and incremental deploys in runner.py
- Add "Install Time" counter to both perf reports
- Add CoreCLR Debug job entry in pipeline YAML
- Add device (ios-arm64) job entries for both Mono and CoreCLR
- Wire iOSRid env var through to MSBuild for device builds
- [TEMP] Disable non-iOS-inner-loop jobs for CI validation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When the dynamically-resolved manifest references SDK packs not yet
propagated to NuGet feeds, fall back to installing without the
rollback file. This avoids CI being blocked by transient feed
propagation delays.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When the rollback file references SDK packs not yet propagated to
NuGet feeds, retry without the rollback file. Matches the fallback
pattern already added to pre.py.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The simulator HelixWorkItem was unconditionally included, even when
iOSRid=ios-arm64. This caused the simulator to receive device RID
in _MSBuildArgs, producing ARM64 binaries that can't install on a
simulator. Add Condition to exclude it from device jobs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…t.py

Consolidate 12 duplicated simulator/device methods in ioshelper.py into
a unified API (setup_device, install_app, measure_cold_startup, cleanup)
that dispatches internally based on is_physical_device. Removes all
if-is_physical dispatch branches from runner.py.

Extract merge_build_deploy_and_startup and _make_counter to module-level
helpers. Inline the incremental iteration loop (was a nested function
with 10 parameters). Simplify post.py to reuse ioshelper instead of
duplicating device detection. Extract inject_csproj_properties in pre.py.

Net reduction: -232 lines across 4 files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ling

Re-apply run_env_vars (including iOSRid) right before perf_send_to_helix()
so the MSBuild .proj ItemGroup conditions can correctly exclude the
simulator work item from device jobs.

Add '|| exit $?' to setup_helix.py PreCommands so that when setup_helix
exits non-zero (e.g., Xcode too old), the Helix shell stops instead of
continuing to run test.py which would fail with a less clear error.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Env var inheritance through msbuild.sh/tools.sh is unreliable for
iOSRid. Add ios_rid field to PerfSendToHelixArgs and pass it as
/p:iOSRid=<value> on the MSBuild command line so it reaches .proj
evaluation deterministically. Also set it via set_environment_variables
as belt-and-suspenders.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Mono_InnerLoop → Mono_InnerLoop_Simulator
CoreCLR_InnerLoop → CoreCLR_InnerLoop_Simulator

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace simctl (simulator) and devicectl (device) install/launch commands
with mlaunch to match the real Visual Studio F5 developer experience:

- Simulator: --launchsim combines install + launch (install_app returns 0)
- Device: --installdev for install, --launchdev for launch
- Device cleanup: --uninstalldevbundleid replaces devicectl uninstall
- Simulator cleanup: unchanged (simctl terminate + uninstall)
- Added _resolve_mlaunch() to find mlaunch from iOS SDK packs

Device detection (devicectl) and simulator management (simctl boot/
terminate/uninstall) remain unchanged. The install_app/measure_cold_startup
API is preserved so runner.py requires no changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Instead of making install_app() a no-op for simulator, use
mlaunch --installsim to get a separate install measurement.
measure_cold_startup() still uses --launchsim for launch timing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…stall

After installing the maui-ios workload, read _RecommendedXcodeVersion
from the SDK's Versions.props and switch to the matching Xcode_*.app
if the currently active Xcode doesn't match. This handles the case
where Helix agents have a newer Xcode than the SDK requires.

Falls back gracefully to the already-selected Xcode if no matching
installation is found.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 6, 2026 07:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 7 comments.

Comment thread src/tools/Reporting/Reporting/Reporter.cs Outdated
Comment thread src/scenarios/shared/ioshelper.py
Comment thread src/scenarios/shared/runner.py Outdated
Comment thread eng/pipelines/sdk-perf-jobs.yml Outdated
davidnguyen-tech and others added 2 commits May 6, 2026 11:47
The 6 JSON files under src/scenarios/mauiiosinnerloop/results/ are
local measurement output produced by run-local.sh (which copies
traces/*.binlog and *-versions.json from a local run for inspection).
They were accidentally committed in earlier work-in-progress commits
and have no place in the PR — production results are uploaded to the
perf-lab via upload.py.

Add a .gitignore entry inside the scenario directory so this can't
happen again.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Previously fix_coresimulator_permissions ran 'sudo chown -R user:staff
/Library/Developer/CoreSimulator', which on Apple Silicon Helix machines
descends into the iOS runtime image at .../Volumes/iOS_NN/... — an
Apple-mounted read-only volume containing hundreds of thousands of
files we have no permission to chown. Each one produced an 'Operation
not permitted' line that was tee'd into the work-item console log,
amounting to ~727k log lines and >200 MB per work item. Both the device
and simulator paths called the function, so device runs (where the
chown is only needed because actool spawns AssetCatalogSimulatorAgent
during 'dotnet build' for ios-arm64) were just as bloated.

Replace the plain 'chown -R' on the system-wide path with a
'find ... -prune -o -exec chown' that excludes the Apple-managed
read-only subtrees (Volumes, Profiles, Cryptex, Images). Per-user
~/Library/... paths still use plain 'chown -R' because they don't
contain those mount points.

Verified the find -prune syntax on macOS against a synthetic tree.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 6, 2026 10:57
The PYTHONPATH change ($PYTHONPATH -> ${PYTHONPATH:-}) is a useful
robustness fix for set -u callers, but it is unrelated to the iOS
inner loop work and affects every dotnet-performance scenario. Keep
the PR focused — the change can land separately on its own merits.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 5 comments.

Comment thread src/tools/Reporting/Reporting/Reporter.cs Outdated
Comment thread src/scenarios/shared/runner.py Outdated
Comment thread src/scenarios/shared/ioshelper.py Outdated
Comment thread src/scenarios/mauiiosinnerloop/post.py Outdated
Comment thread eng/pipelines/sdk-perf-jobs.yml Outdated
davidnguyen-tech and others added 2 commits May 6, 2026 13:58
Previously, when find_and_stage_signing_artifacts() failed to locate
embedded.mobileprovision and the 'sign' tool, setup_helix.py wrote a
SKIPPED.flag sentinel and test.py treated it as a pass (sys.exit 0).
The intent was 'don't fail the build for a known queue provisioning
gap', but the result is exactly the failure mode we don't want: a
green build that hides the fact the scenario didn't actually run.

Hard-fail instead. When signing artifacts are missing, log a loud
'WORK ITEM FAILED — DEVICE INFRA UNAVAILABLE' banner with the exact
reason (which artifact is missing, which roots were searched, what
needs provisioning) and sys.exit(1). The work item turns red, the
console log makes the gap obvious to humans, and ESI can ticket the
queue provisioning work without anyone wondering why the dashboard
shows green.

- setup_helix.py: replace write_skip_sentinel + return 0 path with a
  banner + sys.exit(1); drop the SKIP_SENTINEL constant and the
  write_skip_sentinel helper; update the module docstring.
- test.py: drop _SKIP_SENTINEL and _check_skip_sentinel(); the
  scenario runner now starts immediately under __main__.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Build 2968516's Mono Device job failed with:
  Unable to boot device due to insufficient system resources.
  maxUserProcs: 1333, runningUserProcs: 1221, enforcedProcBuffer: 183

Earlier in the same build the CoreCLR Device job logged 5 leftover
'PerfTest-iPhone-*' simulators in the 'Booted' state from prior workitem
runs. Each booted simulator forks ~150-200 system daemons, so a handful
of leaked simulators is enough to exhaust the per-user macOS process
rlimit before this workitem's own simctl boot can succeed (NSPOSIX 67).

Leaks happen when a previous workitem's post.py crashed before reaching
delete_simulator(), or when the workitem was killed mid-run by a Helix
timeout. Existing post.py cleanup is fine for the happy path; the new
sweep is a defensive safety net at the start of create_and_boot_simulator
that shuts down + deletes any existing PerfTest-iPhone-* simulators.

Conservative scope: only touches devices whose name starts with the
well-known 'PerfTest-iPhone-' prefix (set by _unique_simulator_name),
so unrelated workitems sharing the queue are never disturbed. Best-
effort — failures are logged but don't abort setup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 6, 2026 12:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 5 comments.

Comment thread src/tools/Reporting/Reporting/Reporter.cs Outdated
Comment thread src/scenarios/shared/ioshelper.py
Comment thread eng/pipelines/sdk-perf-jobs.yml Outdated
Comment thread eng/pipelines/templates/build-machine-matrix.yml Outdated
Comment thread eng/performance/maui_scenarios_ios_innerloop.proj Outdated
davidnguyen-tech and others added 2 commits May 6, 2026 14:41
… simulator fallback

Audit triggered by user concern: 'are we falling back to simulator in CI
if the devices aren't available? I do NOT want that'.

Confirmed end-to-end: no path silently downgrades a device job to a
simulator. runner.py:1093 already raises an Exception when no UDID is
detected, and IOS_RID/RuntimeIdentifier are never flipped from ios-arm64
to iossimulator-* anywhere.

But the device branch of setup_helix.main() was sloppy: when
detect_physical_device() returned None, it logged a WARNING and let
setup continue. The work item then wasted ~5 minutes on dotnet build
before runner.py finally raised the cryptic 'Physical device mode
requires a device UDID' exception during install.

Replace the soft warn-and-continue with a hard 'WORK ITEM FAILED — NO
PHYSICAL DEVICE' banner + sys.exit(1) at the same point we fail for
missing signing infra. Same shape, same loud telemetry, same intent:
queue provisioning gaps must show up as red builds with an obvious
reason in the console log, never as silent skips or accidental
simulator measurements masquerading as device numbers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…vice path

The device-job branch was running find_and_stage_signing_artifacts()
first, then detect_physical_device() second. On Mac.iPhone.13.Perf the
signing artifacts are missing, so the work item bails before logging
anything about what physical hardware is (or isn't) attached. Result:
when a human asks 'what iPhone is on that queue?' the only answer in
the logs is 'we never checked'.

Reorder so physical-device detection runs first (its own loud failure
banner) and signing-artifact discovery runs after (its own loud
failure banner). Both gates check independent infrastructure — a
disconnected iPhone vs missing keychain materials — and surface
independently. Now the device log always shows what hardware
'xcrun devicectl list devices' returned, regardless of signing state.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 6, 2026 12:46
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 5 comments.

Comment thread src/scenarios/shared/ioshelper.py Outdated
Comment thread src/scenarios/shared/ioshelper.py
Comment thread eng/performance/maui_scenarios_ios_innerloop.proj Outdated
Comment thread eng/pipelines/sdk-perf-jobs.yml Outdated
Comment thread eng/pipelines/sdk-perf-jobs.yml Outdated
* Consolidate maui_scenarios_ios_innerloop.proj into maui_scenarios_ios.proj
  gated on $(_IsInnerLoop) derived from $(RunKind), matching the Android
  consolidation. Updates sdk-perf-jobs.yml inner-loop entries to reference
  the consolidated proj.
* Drop the 23 "if false # [TEMP] Disabled for iOS inner loop CI validation"
  wrappers from eng/pipelines/sdk-perf-jobs.yml. Restores the existing
  private + scheduled SDK perf matrix that was inadvertently disabled.
* Rename iOSInnerLoopParser to IOSInnerLoopParser (and MetricType enum
  member) to match PascalCase used by sibling parsers; update const.py
  metric mapping accordingly.
* Revert unrelated PERFLAB_BUILDTIMESTAMP fallback change in Reporter.cs.
* Drop the local-only run-local.sh script (not used by CI).
* Make post.py cleanup best-effort: never sys.exit(1), to match Android.
* Remove unused 'import hashlib' in shared/runner.py.
* ioshelper._resolve_mlaunch: pick the iOS SDK pack by parsed version key
  instead of lexicographic sort so 26.10 ranks above 26.9.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 20, 2026 13:39
@kotlarmilos kotlarmilos force-pushed the nguyendav/maui-ios-inner-loop branch from d8b0cc6 to c877a5f Compare May 20, 2026 13:42
…how sudo

- Remove obsolete PublishReadyToRunStripDebugInfo workaround (dotnet/runtime#124604 flowed)
- Split --msbuild-args on ';' instead of replacing with space + shlex (preserves values containing ';')
- Restore edited source files in finally block so reused workspaces start clean
- Run 'log show' with sudo to read the root-owned logarchive
- Drop simulator fallback to ios-arm64 bundle search (avoid picking up stale device builds)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Comments suppressed due to low confidence (1)

src/scenarios/shared/runner.py:252

  • testtypes (from shared/testtraits.py) is used in the "Please specify a test type" error message, but it doesn't include the newly added iosinnerloop command. Consider adding const.IOSINNERLOOP to shared/testtraits.py:testtypes so users get an accurate list of available subcommands.
        iosinnerloopparser = subparsers.add_parser(const.IOSINNERLOOP,
                                                 description='measure first and incremental build+deploy time via binlogs (iOS)')
        iosinnerloopparser.add_argument('--csproj-path', help='Path to .csproj file to build', dest='csprojpath')
        iosinnerloopparser.add_argument('--edit-src', help='Modified source file paths, semicolon-separated', dest='editsrc')
        iosinnerloopparser.add_argument('--edit-dest', help='Destination paths for modified files, semicolon-separated', dest='editdest')
        iosinnerloopparser.add_argument('--framework', '-f', help='Target framework (e.g., net11.0-ios)', dest='framework')
        iosinnerloopparser.add_argument('--configuration', '-c', help='Build configuration', dest='configuration', default='Debug')
        iosinnerloopparser.add_argument('--msbuild-args', help='Additional MSBuild arguments', dest='msbuildargs', default='')
        iosinnerloopparser.add_argument('--bundle-id', help='iOS bundle identifier', dest='bundleid')
        iosinnerloopparser.add_argument('--device-id', help='iOS device ID (UDID for physical device, simulator ID or "booted" for simulator)', dest='deviceid', default='booted')
        iosinnerloopparser.add_argument('--device-type', choices=['simulator', 'device'], help='Target device type: simulator (default) or physical device. Auto-detected from RuntimeIdentifier if not set.', dest='devicetype', default=None)
        iosinnerloopparser.add_argument('--inner-loop-iterations', help='Number of incremental build+deploy+startup iterations (1+)', type=int, default=10, dest='innerloopiterations')
        self.add_common_arguments(iosinnerloopparser)

        args = parser.parse_args()

        if not args.testtype:
            getLogger().error("Please specify a test type: %s. Type test.py <test type> -- help for more type-specific subcommands" % testtypes)
            sys.exit(1)
  • Files reviewed: 19/19 changed files
  • Comments generated: 2

Comment thread src/scenarios/shared/runner.py Outdated
Comment thread src/scenarios/mauiiosinnerloop/test.py Outdated
kotlarmilos and others added 2 commits May 20, 2026 16:01
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
iPhone13.Perf fails identically to iPhone17.Perf on main (AppleSdkSettings
..cctor ArgumentNullException at dotnet publish NetiOSDefault.csproj).
Keep matrix aligned with main; pool fix is a dnceng infra issue, not
in scope of this PR.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 20, 2026 14:45
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 18/18 changed files
  • Comments generated: 2

if self.framework:
base_cmd.extend(['-f', self.framework])
if self.msbuildargs:
base_cmd.extend([arg for arg in self.msbuildargs.split(';') if arg])
Comment on lines +97 to +100
m = re.search(r'Microsoft\.iOS\.Sdk\.([^/\\]+)', p)
if not m:
return ()
parts = re.split(r'[.\-+]', m.group(1))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants